Pyspider Deploment(RabbitMQ+MongoDB)

### config.conf
  {
    "taskdb": "mongodb+taskdb://127.0.0.1:27017/taskdb",
    "projectdb": "mongodb+projectdb://127.0.0.1:27017/projectdb",
    "resultdb": "mongodb+resultdb://127.0.0.1:27017/resultdb",
    "message_queue": "amqp://127.0.0.1:5672/%2F",
    "webui": {
      "username": "root",
      "password": "root",
      "need-auth": true
    }
  }

注意，这里的MongoDB跟RabbitMQ都默认没有密码。
如需要用户名密码。

"taskdb": "mongodb+taskdb://username:passwd@127.0.0.1:27017/taskdb",

运行

# start **only one** scheduler instance
pyspider -c config.json scheduler
# phantomjs
pyspider -c config.json phantomjs
# start fetcher / processor / result_worker instances as many as your needs
pyspider -c config.json --phantomjs-proxy="localhost:25555" fetcher
pyspider -c config.json processor
pyspider -c config.json result_worker
# start webui, set `--scheduler-rpc` if scheduler is not running on the same host as webui
pyspider -c config.json webui